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Timing Is Everything: 

Temporal Variation and Measures of School Quality 

Peter T. Goff, Jihye Kam, and Jacek Kraszewski 


Surveys are a well-established tool used to expand our insight into the mechanics of 
organizations and the behaviors of people (Ary, Jacobs, Razavieh, & Sorensen, 2010). The 
analysis of the large-scale, nationally-representative datasets, such as those sponsored by the 
National Center for Education Statistics (NCES) and the National Science Foundation (NSF), 
assist in supporting and developing federal, state, and local legislations, regulations, and policies 
in education. Researchers collect, analyze, evaluate, and convey statistical data related to 
performance evaluations of students, parents, teachers, staffs, and principals. These national 
datasets, such as the Early Childhood Longitudinal Study (ECLS), the Schools and Staffing 
Survey (SASS), the Baccalaureate and Beyond (B&B), and the National Survey of College 
Graduates (NSCG), span early childhood education to postgraduate labor market outcomes. 
Practitioners increasingly use surveys to better understand organizational climate (e.g., 

Halverson, Kelley, & Shaw, 2014), leadership behaviors (e.g., Goldring, Porter, Murphy, Elliott, 
& Cravens, 2009), and instructional practices (e.g., Balch, 2012). 

Perspectives and Perceptions 

In the current era of accountability, educators, practitioners, and parents rely, in part, on 
survey measures to infonn summative evaluation, formative development, and school selection 
decisions. Therefore, the validity and reliability of survey responses regarding school 
performances are critical to the success of market-based educational refonns (Downey, Hippel, 

& Hughes, 2008). When the survey data are applied in educational practice, the temporal 
differences in the survey timing may generate problems. For example, schools using surveys to 
measure developmental progress typically examine how a score in the current year (/) differs 
from a score in the prior year (t-1). If the survey timing matters, the growth is over or 
underestimated and its true magnitude remains unknown. If fluctuation in timing is idiosyncratic, 
then this over or underestimation is simply measurement error. However, if the survey timing is 
correlated with substantive factors of interest, then temporal variation can be a source of bias, 
undennining decisions and inferences drawn from the data collected. Given the highly cyclic 
nature of schools and schooling, we might expect that temporal effects—to the extent that they 
exist at all—might be more pronounced in an educational setting. 

In acknowledgement of the complex and multifaceted nature of education, some school 
districts have moved away from a strictly test-based vision of accountability, expanding 
performance measures to incorporate the views and perspectives of individuals closely involved. 
Polk County Public School District in Florida, for example, has developed a leadership 
evaluation system that integrates a measure of learning-centered leadership, as measured through 
the Vanderbilt Assessment of Leadership in Education (VALEd) (Porter et ah, 2008). A similar 
system is found in Baltimore, Maryland. The performance measures regarding student and parent 
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perspectives on school climates, as well as teacher instructional effectiveness captured by the 
Measures of Effective Teaching Study (MET), are used in the Baltimore school leader 
effectiveness evaluation (Kane & Staiger, 2012). Given such prevalence of survey measures in 
education, these measures have to depict an accurate portrayal of differences across schools and 
over time. 

Validity and Reliability 

The psychometric considerations of survey research are well developed and widely discussed 
in the literatures of psychology and are primarily clustered around the concepts of validity and 
reliability; however, there is little attention paid to the temporal effects of the survey timing on 
participants’ responses. Validity refers to “the degree to which evidence and theory support the 
interpretations of test scores entailed by the proposed uses of tests,” while reliability refers to 
“the degree to which an assessment tool produces stable and consistent results” (American 
Educational Research Association, American Psychological Association, National Council on 
Measurement in Education, 1999). In particular, the “internal consistency” or “item 
homogeneity,” which denotes “the reliability of a scale based on the degree of within-scale item 
inter-correlation,” has been long used in psychometric testing (Crobanch, 1951). One factor 
typically omitted from the psychometric testing pertains to the exogenous temporal variation, 
even though some psychometric papers directly address this feature (Buhyoff & Wellman, 1979; 
Kitamura, Shima, Sugawara, Toda, 1999; Cole & Stewart, 2002; Young, Blodgett, & Reardon, 
2003; Murray, 2003). 

Our study aims to contribute to the literature by examining how the survey timing in tenns of 
day of week, season, and proximity to high-stakes exam is systematically related to teachers’ 
responses regarding their perceptions of students’ poverty and behavior, their control in the 
classroom, student behavior, and their principal’s leadership support. The study is organized as 
follows: Section 1 presents the literature review, while Section 2 and Section 3 provide 
identification strategies and research methodology, respectively. Section 4 highlights our main 
findings and Section 5 discusses the findings and possible directions for further research. 

1. Literature Review 

Several scholarly streams of thought in the literature focus on the instability of survey 
responses. Previous research on the instability of survey responses has empirically investigated 
specific factors such as survey mode and monetary rewards affecting survey responses while 
attempting to hold all other potential variables constant. We incorporate our hypothesis of the 
temporal variations in the perceptions of teachers with the methodological issues in the 
instability of survey responses. For simplicity, the related literatures are categorized under two 
general classifications: (1) antecedents and consequences of temporal variations, and (2) the 
seasonal variation in teacher perceptions. 
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Antecedents and Consequences of Temporal Variations 

Researchers have long recognized the psychological antecedents and consequences of human 
behaviors with respect to temporal variation. The literatures in psychology have shown distinct 
patterns in human behavior associated with temperature and other weather variables such as 
hours of sunlight, duration of daylight, humidity, and air pressure. Suicides, for example, are 
more likely to happen in spring and summer than in winter (Chew & McCleary, 1995; Ajdacic- 
Gross et al., 2006). This is in line with a positive effect of seasonal oscillations on crime rates. 
Warmer temperatures affect the rates of violent behaviors such as robbery (DeFronzo, 1984; 
Anderson & Anderson, 1984), homicide (Michael & Zumpe, 1983a, 1983b; McDowall & Curtis, 
2014; Morken & Linaker, 2000), and sexual assault (Perry & Simpson, 1987; McLean, 2007). In 
particular, significant variations in violent incidents were observed May through June and 
October through November (Morken & Linaker, 2000). Violent behavior also peaked on 
Sundays and Mondays (Sisti, Rocchi, & Preti, 2012; Rastogi et al., 2013). 

These temporal variations of crime revealed variations of crime opportunities (Carlsmith & 
Anderson, 1979) or emotional changes (Hipp, Bauer, Curran, & Bollen, 2004) related to 
personality disorders and moral responsibilities, which are defined as Seasonal Affective 
Disorder (SAD), which is a cyclic illness characterized by recurrent episodes of depression in 
fall and winter months alternating with periods of normal or mild moods in summer and spring 
months (Rosenthal, 1987, p. 57). There is also compelling evidence of a link between seasonal 
depression and seasonal variations in stock returns (Kamstra, Kramer, & Levi, 2003) and 
consumption expenditures (Barrow & McGranahan, 2000). Thus, various aspects of human 
behavior are significantly influenced by temporal variations. 

Organizations also exhibit a predictable ebb and flow of behavior over time. Tax firms have 
substantial demands on their time and resources during the first half of the year and comparably 
more flexibility in the third quarter. Hotels, restaurants, and other organizations tied to tourism 
oscillate between periods of frenetic chaos and relative calm across days, weeks, and years 
(Baxter & King, 1999). Schools operate on a particularly well-known and established cycle: 
Annual schooling is regularly punctuated with breaks in the winter and spring, with a longer 
(typically 3 month) hiatus during the summer. The statewide standardized, high-stakes 
accountability tests that often inform and dictate school reform efforts are typically offered in 
early spring. Recent research has shown that effective human-resource management strategies in 
education are tied tightly to the school calendar (Drake et al., 2014). 

The school cycle can dictate how a teacher copes with the demands of the job, and this may 
be particularly evident among new teachers (Weiss, 1999). During the early weeks of school, 
teachers establish routines and instructional norms, while also scheduling meetings with parents, 
administrators, and other teachers. As the school year progresses collaborative interactions may 
drop, instructional demands accumulate, and evaluations begin. In late winter and spring, teacher 
burnout increases (Brouwers & Tomic, 2000), in part because teachers cast doubt on their 
professional abilities, which is related with low self-efficacy (Femet, Guay, Senecal, & Austin, 
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2012). At the end of the school year, teachers focus more on summative assessment and 
evaluation of student learning (Carson, 2006). 

Thus, we see that both organizations and people are subject to systematic temporal 
fluctuations, and these fluctuations may be of particular note within schools. 

Seasonal Variation in Teacher Responses 

A seasonal perspective on learning outcomes has been discussed since Hayns (1978, 1987) 
found a notably larger achievement gap during the summer months rather than the rest of school 
year. The author explained this gap by addressing non-school factors developed during the 
summer break. The subsequent literatures in the seasonality of learning outcomes have since 
expanded substantially but most have retained a narrow focus on summer learning gains or losses 
(Cooper, Nye, Charlton, Lindsay, & Greathouse, 1996; Entwistle & Alexander, 1992, 1994; 
Alexandar, Entwisle, & Olson, 2001). We have found that inquiry regarding the temporal effects 
of teachers’ perceptions of leadership support and student behaviors is comparatively limited. 
This study contributes to the literature by examining seasonal effects on the perceptions of 
teachers regarding students’ poverty and behavior, their control in the classroom, and the 
principal leadership across the school year. This paper discusses survey-based research, practice, 
and policies by identifying and quantifying biases related to temporal variations of the timing of 
survey administration. 


2. Identification Strategies 


Data Description 

The SASS is administered to a sample of elementary and secondary schools representative of 
national and state levels. These surveys, sponsored by the Institute of Education Sciences, were 
first administrated in the 1987-88 school year and have been re-administered with minor 
changes six other times over the past 25 years. The survey questionnaires represent a wide range 
of topics from teacher demand, teacher and principal characteristics, general conditions in 
schools, principals’ and teachers’ perceptions of school climate and problems in their schools, 
teacher compensation, district hiring, and retention practices to basic characteristics of the 
student population. SASS information is obtained through four questionnaires: The school 
questionnaire, the teacher questionnaire, the principal questionnaire, and the school district 
questionnaire. Among them, we focus on four factors: teachers’ perceptions of students’ poverty, 
student behavior, teacher control in the classroom, and teachers’ perceptions of leadership 
support using the responses from the teacher questionnaire. The sample of this study consists of 
2007-08 SASS, which is the most recent available for analysis. It is composed of 38,240 
teachers of 7,931 schools in 4,613 districts across the United States. 1 We restricted the sample to 
fulltime teachers. 


1 For a detailed description of the sampling design used in the SASS, see Tourkin et al. (2010). 
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The SASS administrators send out a framework questionnaire to schools once schools have 
been selected for sampling. The questionnaire asks schools for infonnation on teachers in order 
to create a sampling frame for the selection of teachers within the school. The framework 
questionnaire is submitted to the SASS administrators whereas the teacher questionnaires are 
delivered to a volunteer survey coordinator for each school. The coordinator distributes the 
questionnaires to selected teachers. The teachers ensure anonymity and confidentiality of their 
information. The responses are submitted to the SASS administrators, including information 
regarding the beginning and completion dates of the survey. 

Conceptualizing Temporal Variation in Schools 

Schools are organizations that maintain a regular and predictable schedule in accordance with 
the school district policy and procedures. The common academic year calendars are established 
for each semester while a detailed curriculum and instruction are designed on a weekly basis. 

The workload burdens on teachers vary across the days of week based on the intensity of the 
daily curriculum and instruction. In addition, the tempo and pulse of the school year from late 
August to early June provides an opportunity for researchers to study how perceptual measures 
relate to a specific season or time of year. For instance, teachers may be more likely to perceive 
student poverty as a serious problem in winter rather than in spring because the student poverty 
rates can be more easily revealed by students’ clothing. 

When exploring the role of temporal variations in survey measures, we identified three 
aspects of time that may be most relevant. First, we consider the role that day of the week may 
play, hypothesizing, for example, that responses on Fridays may be systematically different from 
responses collected on Tuesdays. Our second perspective of time looks across the entire school 
year from October 2 to June. Lastly, we consider temporal variations surrounding high-stakes 
exams, which typically take place during a 3-week window in the spring. This hypothesis 
demonstrates the challenges and stresses that teachers encounter when preparing their students 
for high-stakes exams (Kruger, Waddle, & Struzziero, 2007), suggesting that school supports, 
teacher perspectives, and organizational practices in the weeks leading up to the exams may 
differ from the weeks following the exam. 

As we conceptualize variation in perceptual data over time we identify two sources of 
variation that may be of interest to researchers and policy makers: “substantive variation” (owing 
to systematic variation in the substantive construct of interest) and “endogenous variation” 

(owing to systematic variation in temporal factors related to respondent perception but unrelated 
to the underlying construct). As an example of the first, when examining student behavior over 
the course of the school year, we may find that students behave differently during the initial 
weeks of school as compared to later in the year after classroom norms and routines have been 
established. As an example of the second, we might envision a scenario where systematic, 


2 Although many schools begin in August or September, the survey responses do not begin until October. This gives 
teachers ample time to acclimate to the school, students, and peers prior to submitting any surveys. 
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external pressures (the approaching high-stakes state exams, for example) cause the teacher to 
perceive student behavior to be more (or less) orderly than the “true” measure of student behavior. 

Two plausible hypotheses facilitate an inquiry into these temporal variations in schools. The 
first suggests that people’s perspectives are sensitive to personal, professional, or environmental 
influences. Under this hypothesis, we may expect to observe structural variations in survey 
responses over time, even when the underlying construct we are measuring remains constant. To 
investigate the prevalence of perceptual change within schools we selected two measures that we 
expect to remain constant over time: student poverty and teachers’ classroom control. The logic 
here is that any systematic changes in the perception of these factors are likely not related to the 
underlying construct, but rather are related to external, time-relevant factors. 

A second hypothesis suggests that organizational factors within schools change over the 
course of the school year, and they do so in common, predictable ways. To investigate this 
second hypothesis we selected two measures that might vary systematically over time: teachers’ 
perceptions of student behavior and teachers’ perceptions of leadership support. The four 
measures we selected are discussed in greater detail below. 

Teachers’ Perceptions 

Student poverty. This item captures the extent to which teachers perceive student poverty to 
be a problem in the school. The perceptions of teachers are coded as an ordered categorical 
variable with four responses: “serious problem” (1), “moderate problem” (2), “minor problem” 
(3), and “not a problem” (4). To easily interpret the results, the poverty scale has been recoded 
such that student poverty is regarded as a serious problem as the scale increases. Although 
student poverty varies between and within schools over multiple years, we have found no 
research to suggest systematic variation in student poverty during the school year. If student 
poverty is a time-invariant or a more flexible construct that varies only stochastically relative to 
the school year, any systematic variation in this measure can reasonably be attributed to 
perceptual variation. More detailed information on this measure is presented in Table A.3 of 
Appendix A. 

Control in the classroom. To measure teachers’ control in the classroom, we use a set of 
items that solicited teachers’ perspectives of control over six processes: selecting textbooks and 
other instructional materials; selecting content, topics, and skills to be taught; selecting teaching 
techniques; evaluating and grading students; disciplining students; and determining the amount 
of homework to be assigned. The perceptions of teachers regarding their control in the classroom 
are coded as an ordered categorical variable with four responses: “no control” (1), “minor 
control” (2), “moderate control” (3), and “a great deal of control” (4). The Cronbach’s alpha is 
0.72, which is on par with the accepted reliability standard of 0.70 (Hair, Anderson, Tatham, & 
Black, 1995; Nunnally, 1978). As with student poverty, we expect teacher control in the 
classroom to vary between schools (Weiss, 1993) as well as within schools over multiple years; 
however, we have little reason to believe these measures of classroom control vary within 
schools over a single school year. Table A.4 of Appendix A presents the related statistics. 
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Leadership support. We use three items to measure teachers’ perceptions of leadership 
support. The perceptions of teachers on leadership support are coded as an ordered categorical 
variable with four responses: “strongly agree” (1), “somewhat agree” (2), “somewhat disagree” 
(3), and “strongly disagree” (4). To easily interpret the results, we recoded the scales in opposite 
direction. The Cronbach’s alpha is 0.728. Teachers’ perceptions of leadership support are tied to 
factors such as teacher efficacy and retention (Boyd et al, 2011). We hypothesize that leadership 
support is a construct that may evolve over the course of the school year as the demands on the 
principals’ time changes and the principals’ relationship with teachers, particularly newly hired 
teachers, develops. The relevant statistics are viewed in Table A. 5 of Appendix A. 

Student behavior. Our final perceptual measure pertains to teachers’ perspectives of student 
behavior, asking teachers to document the extent to which tardiness, absenteeism, and class¬ 
cutting are problematic in their school. The perceptions of teachers regarding student behavior 
are coded as an ordered categorical variable with four responses: “serious problem” (1), 
“moderate problem” (2), “minor problem” (3), and “not a problem” (4). The scales are recoded 
in opposite direction for a consistent interpretation. The Cronbach’s alpha is 0.839. Teachers’ 
perceptions of student behavior in winter and summer are also significantly different from fall (p 
= 0.000 < 0.001). In contrast, perceptions of student behavior are not significantly different 
before and after a high-stakes exam (p = 0.194> 0.1). While we were unable to document any 
prior work that empirically identifies temporal trends in student behavior, anecdotal evidence 
suggests that tardiness, absenteeism, and class-cutting are likely to become more prevalent as the 
school year progresses. 

Data verification. To examine how survey measures differ before and after high-stakes 
exams, we searched the state testing dates for each state in the 2007-08 school year by utilizing 
the Internet Archives Wayback Machine. We restrict our subsample for the temporal variations 
in terms of the proximity to high-stakes exams to the state comprehensive assessments in Grades 
2-8. That means high school teachers are excluded in our analysis. The dates of the testing 
windows obtained from each state’s department of education websites were matched with 
information provided by the Council of Chief State School Officers (CCSS). 

A large portion of responses were submitted between late October and December before the 
winter break in the entire sample, whereas the average testing window for spring exams is 
between March 10th and April 1st, 2008. We included only states offering a high-stakes exam 
during fall semester when the majority of SASS participants submitted their responses and 
excluded Oregon because of its wide range of testing window (Table A.2 of Appendix A). In the 
state test sample, the average testing window for fall exams is between November 29th and 
December 18th, 2007 (Figure 1). 
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Figure 1. Distribution of Response Dates: Full Sample (Left) and Fall Testing (Right) Sample 



Note: A kernel density plot shows the distribution of teacher survey dates across the school year. 


There are two important considerations to be aware of here. First, we were unable to find the 
state test dates in North Carolina and thus North Carolina was dropped from the model. Second, 
the state test dates in the 2007-08 school year for Indiana, Iowa, North Dakota, New Hampshire, 
Rhode Island, Utah, Virginia, and Wisconsin could not be ascertained, so we assigned the testing 
dates based on the test windows of these states in years before and after. 

3. Methodology 

This study addresses the effects of the survey timing on the teachers’ perceptions of their 
students (behavior and poverty), their principal, and their classroom. Regression analysis is used 
to explore the relationship between temporal variation in responses with respect to day of week, 
season, and proximity to high-stakes exams and the perceptions of teachers regarding students’ 
poverty and behavior, their classroom control, and the principal leadership. This section outlines 
the methods we applied to discuss three phenomena of temporal variations: variations pertaining 
to day of week, variations across the entire school year, and variations before and after a 
statewide exam. When examining each of these three aspects of temporal variations, we use the 
four perceptual measures previously outlined: student poverty, classroom control, leadership 
support, and student behavior. To facilitate interpretation of our findings, we standardize the 
scores for each teachers’ perception to have within-sample mean 0 and variance 1 after taking 
mean values over the respective sets of scales (Heckman, Stixrud, & Urzua, 2006). 

We use several aspects of the survey administration process to strengthen our identification 
strategy. To facilitate the data collection process, the SASS begins the survey administration in 
September and staggers the deployment of its remaining surveys across the school year. 
Although this staggered timing is not random, it is not purposefully related to factors such as 
geography, urbanicity, or school quality. Further variation in timing manifests within schools, as 
the schools are given surveys for all selected teachers within a school at the same time, yet some 
teachers take longer than others to complete the survey. We take advantage of both within and 
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between-school variation in the survey timing to estimate the impact of timing on survey 
responses. 

When exploring the role of temporal variations across the school year in terms of day of 
week, season, and proximity to high-stakes exams, we use several approaches to mitigate biases 
that may result from endogenous relationships between response time and teacher characteristics. 
We first control for observable characteristics at the school and teacher levels, because teachers 
who promptly complete their survey may have systematically different views from those of their 
peers who wait to complete the survey. A second strategy for limiting endogenous variation 
focuses on an examination of temporal variation using only the teachers from each school who 
responded first. The logic of this approach is that “first responders” might be systematically 
different from their colleagues who respond later, and an examination of response patterns 
among first responders will yield a cleaner estimate of temporal variation. In a third approach, 
we use the date the first responder submitted the survey as an instrument for completion date for 
all other teachers within the school. This approach is based on the assumption that the 
completion date of the first responders will be related to the completion date of other teachers in 
the school, but unrelated to unobserved factors that may be correlated with later responses; first 
responders are not included in this analysis. To conduct instrumental variable (IV) estimations, 
the instruments should be uncorrelated with errors but partially correlated with the endogenous 
treatment variable once the other explanatory variables are controlled. The R-squared and the 
adjusted R-squared values of the first-stage regression are around 0.882 for estimations in the 
seasonal variations, ruling out potential concerns regarding weak instrument bias (Stock & Yogo, 
2005). The F-statistics of for the seasonal variation IV analyses exceed 600, considerably larger 
than the minimum rule of thumb value of 10. 

We use these various approaches to examine the four survey scales identified above, 
including two that are expected to exhibit substantive variation—leadership support and student 
behavior—and two that exhibit perceptual variation: perceptions of poverty and classroom 
control. 

Analytical Approach 

As indicated, we examine three sources of temporal variation: day of week, season, and 
before and after high-stakes state exams. For each scenario, the dependent variables are four 
measures of teachers’ perceptions: their instructional challenges related to poverty, classroom 
control, leadership support, and student behavior (see Table A.l in Appendix A). For many of 
our models we present results with and without school and teacher control variables. Models 
without control variables may be of interest to practitioners who use raw, unadjusted survey 
means, while researchers who frequently integrate control variables may have greater interest in 
the covariate adjusted regression estimates. School control variables include urbanicity dummies 
(city, suburb, town, and rural), total enrollment in school, program type dummies (regular, 
special program emphasis, special education, vocational education, and alternative program), the 
percentage of students with an individual education program (IEP) and limited English 
proficiency (LEP), charter school dummy, regional dummies (Northeast, Midwest, South, and 
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West), and school level dummies (primary, middle, high, and combined). Teacher control 
variables include Hispanic ethnic origin dummy, racial dummies (White, Black, Asian, Pacific 
Islander, and American Indian), total years of teaching experience and its squared, gender 
dummy, educational dummies for highest degree earned (associate’s degree or no college degree, 
bachelor’s degree, master’s degree, educational specialist or certificate of advanced graduate 
studies, and doctoral degree), union dummy, and dummies for grades of students. A descriptive 
overview of the full sample and the restricted test sample are presented in Table A.7 and Table 
A. 8 of Appendix A. 

To analyze the impact of daily variation on teachers’ perceptions, we estimate four equations 
of the form shown below (one for each of the teacher perception variables). We denote Yq as the 
perceptions of teacher /' in school j. The model for Scenario 1, day of the week, is specified 
below: 


Y i,j 


— a t j + fii jDay of Week t j + YjSchoolj + SiTeacheri + ( 1 ) 


where Day of Weekij are six dummy variables for day of week, with reference to Friday. 
Schoof and Teacheri are the school and teacher control variables, respectively, as outlined in 
the previous paragraph. 

Similarly, the IV models for Scenario 2, investigating variation across the school year are 
below: 

Zij — a t j + fijSeasoriij + YjSchoolj + SiTeacheri + pjFirst Response j + (2) 


Y t j — a t j + jSeasoni j + YjSchoolj + 


SiTeacheri + AjZj + 


where Seasoni j are dummies for spring and winter seasons, in reference to fall. These variables 
do not pertain to seasons directly but instead represent periods of time that may be most pertinent 
to schools: a fall period running from September through December, a winter period from 
January to March, and a spring period from March to June. 

Zi j is operationalized as the number of days from September 1, 2007; in the same way 
First Response j, our excluded instrument, is the number of days since September first for the 
first respondent in each school. Thus, we are using the response time from the first responders in 
each school to create an exogenous estimate of the response time for all other individuals in the 
school. 

The models for Scenario 3, investigating how proximity to state exams may influence survey 
response patterns, are shown below: 
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Yij — a.i j + Pi jPrwri j + YjSchoolj + SiTeacheri + AiTime Priori + e i,j ( 3 ) 


where Priori j is 1 if a teacher i responds to the survey before the state exam and 0 after the state 
exam. The variable TimePrior indicates the number of days before or after the testing window 
that a survey was completed. As with Scenario 1, we present results for a base model and a 
covariate-adjusted model. 


4. Findings 

The findings from our regression analyses show that temporal variation may introduce 
unwanted biases into survey responses. We find evidence that teachers’ perceptions are sensitive 
to temporal variation, even when the underlying construct of interest remains constant. We also 
find evidence that these constructs themselves—commonly used in formative constructs and 
increasingly used in high-stakes evaluations—show some systematic variation over time. This 
section outlines the evidence supporting these findings, drawing from the regression models 
outlined above. 

Weekly Variation 

Scenario 1 investigates whether the responding day of week affects the survey responses. Our 
findings are presented graphically in Figure 2, which depicts the predicted values from the 
covariate-adjusted model. When we examine the two measures used to examine perceptual 
changes (poverty and classroom control), we see little variation across days, although teachers 
were more likely to report poverty to be an instructional barrier on Mondays rather than on 
Fridays. When we examine the two measures intended to measure changes in constructs— 
leadership support and student behavior—we see that teachers are more likely to view principals 
as supportive on Sundays and Tuesdays (relative to Fridays), and student behavior is reported to 
be lower on Mondays. Full results are presented in Table B. 1 of Appendix B. Similar trends 
among the coefficients are noted when the sample is reduced to only first responders (see Table 
B.2 of Appendix B); however, the 75% reduction in sample size erases much of the statistical 
significance. In sum, these results suggest that surveys completed on Mondays may yield lower 
ratings of perceptions regarding students, and surveys completed on Fridays may yield lower 
ratings of perceptions regarding school leadership. 
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Figure 2. Predicted Values for Day of the Week (Scenario 1) 



0 . 


SUN UON TUE WED THU FRJ SAT 
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Seasonal Variation 

In the second set of our analyses, we examine variation in survey responses across the school 
year. The findings from the IV estimations of the covariate-adjusted models are presented in 
Figure 3. The top two panels represent the two constructs we hypothesized to be constant across 
the school year: teachers’ perceptions of poverty related problems and classroom control. The 
lower two panels represent the two constructs we predicted may vary across the school year. 
Teachers’ perceptions of instructional challenges related to poverty, as well as the aspects of 
classroom control, do not demonstrate significant change across the school year in any of the five 
models we specified (see Table B.3 in Appendix B). The consistency of predicted values for 
student poverty-related challenges and classroom control suggests that teachers’ perspectives are 
not strongly influenced by seasonality. This, in turn, implies that any measurable changes that 
occur across the school year are likely attributable to changes in the underlying construct of 
interest. 

The two constructs in the bottom panel of Figure 3, leadership support and student behavior, 
show notable variation across the school year. Teachers’ perceptions of leadership support is 
greatest at the start of the school year, declines through March before dropping sharply at the end 
of the school year. This result is evident in the base model, the model with school and teacher 
covariates, and the model with covariates that limited the sample to first-responders. Although 
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the same pattern is evident graphically when using predicted values, temporal variation in 
leadership support is not significant in the IV models. Teachers’ perceptions of student behavior 
problems surrounding truancy, tardiness, and absenteeism increase at a modest, though 
consistent, rate throughout the course of the school year. In particular, teachers are more likely to 
negatively perceive student behavior in spring season (Table B.6 of Appendix B). These findings 
are robust across all model specifications. 

Figure 3. Predicted Values across the School Year (Scenario 2) 




Variation Before and After State Exams 

Our final inquiry investigated the extent to which the pressures of high-stakes testing may 
substantially change perspectives and behaviors within schools. Figure 4 presents the min-max 
range of predicted values for each day generated from our covariate controlled models across 
each of our four survey measures. The x-axis shows the number of days before and after the 
exam period. For the three survey measures of student poverty-related challenges, leadership 
support, and student behavior, the covariate-controlled models yielded null results, suggesting 
that measures collected before or after testing are not significantly different. In contrast, teachers 
who responded to the survey prior to the high-stakes test are somewhat more likely to report 
greater classroom control (Table B.7 of Appendix B). These results are substantively unchanged 
when we limit the sample to first responders. 
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Figure 4. Predicted Values Pre- and Post-testing (Scenario 3) 






5. Discussion and Further Research 

The findings from this research are threefold. First, teachers’ report stronger leadership 
support and more challenging student behaviors early in the week. Second, substantive 
constructs appear to change predictably over the course of the school year, particularly declines 
in student behavior and, to a lesser extent, in leadership support. Third, despite the publicity 
given to the pressures arising around high-stakes state exams, we find little evidence that 
responses before the exam are systematically different than those before. This last point comes 
with the caveat that responses to low-stakes surveys, such as the SASS may differ from 
responses to high-stakes surveys. 

For state and district-level practitioners these findings imply that reasonable latitude may be 
given to schools to choose survey dates that best complement local schedules, such as 
professional development days. Under such conditions, small timing differences in survey 
administration should not bias comparisons among schools. 

Although this paper focused on the method of data collection rather than the survey 
constructs directly, school leaders may seek to modify their practice in response to the above 
findings. Specifically these findings suggest that teacher perceptions of leadership support lags 
as the year progresses. Perhaps this is in response to greater responsibilities for both teachers and 
principals later in the year. If this were true, principals may be more likely to attend to other 
demands and their support may flag. It may be that levels of principal support remain fairly 
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constant over time, yet teachers feel more stress later in the year and need principals to ramp up 
their support accordingly. The above findings may also highlight a shifting of the principals’ role 
within an accountability framework. Early in the year principals may adopt the role of coach or 
mentor as they work with teachers to build instructional capacity. As the year progresses, 
principals’ role may shift to that of evaluator as they engage with teachers in a more summative 
manner (e.g., formal observations of classroom teaching). This shift from coach to evaluator may 
push some teachers to see principals as less supportive during the spring than in the fall. 

The decline in perceived leadership support may also arise as a result of the accumulation of 
stress and challenges encountered throughout the school year. With summer is traditionally 
viewed as a period of rest and renewal for teachers it is reasonable for teachers to begin the year 
positive and optimistic and to have this positivity wane when faced with mounting adversity. 

This may be the case even in the absence of any discemable differences in leadership behaviors. 
Whether the decline lies in actual changes in leadership support or is strictly an artifice of 
perception, the message for school leaders is clear: teachers need more support as the school year 
progresses. Focused support of new and novice teachers during late spring may be a particularly 
strong investment on the part of school leaders as these teachers are more likely to leave the 
school or profession. 

Although this research was not designed to provide a comprehensive portrayal of all 
perceptions elicited by teachers, it does provide some direction to leaders as to where their 
support may be needed. As teachers report student behavior to be more problematic as the year 
progresses, principals can focus a portion of their efforts to maintain clear and consistent 
behavioral expectations for students. Especially in elementary schools, the establishment of 
behavioral norms and values is a priority early in the school year. Principals can support teachers 
by ensuring that this focus remains strong through the winter and spring months as well. As 
novice teachers report greater challenges with classroom management, school leaders may want 
to develop strategies to ensure early career teachers have the supports they need to limit 
behavioral challenges and focus on instruction. The research we have presented here suggests 
that it is important for principals to maintain these behavioral supports through the end of the 
school year. 

Researchers engaged in survey research may also benefit from these findings. Following the 
watershed research on the summer learning loss, several studies have structured their research 
design to collect data including achievement testing at the beginning of the school year and 
conclude in the late spring, such as the Early Childhood Longitudinal Study, Kindergarten 
cohort. For designs with a treatment and control group, any seasonal changes would be equally 
manifest in the control condition and are not a concern. However, studies investigating a given 
policy or intervention that lack a control or comparison group may wrongly attribute fall-to- 
spring differences to program effects rather than seasonal change. 

We see three limitations to our study. First, the findings apply only to the measures included 
here; other constructs operationalized through other measures may experience more or less 
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seasonal change. We see this possibility as a fruitful line of subsequent inquiry. Second, most 
teachers responded to surveys during the fall semester while most state exams were conducted in 
spring. This causes a notable reduction in sample size and subsequent reduction in precision. The 
last limitation arises because the testing window period varies across states, making it difficult to 
precisely identify teachers’ perceptions before and after the state exam. In any future work, 
district-level exam dates should be integrated to better estimate how high-stakes exams may 
impact measures of climate, leadership, and efficacy. 
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Table A.l. Dependent Variable Constructs 


\z” triable 

Question Response Options 

Recode 

Cronbach’s 

alpha 

(1) Serious 

To what extent is each of the problem 

Student _ , following a problem in this (2) Moderate 

Poverty ^ ' school? problem 

h. Poverty (3) Minor problem 

(4) Not a problem 

Yes 

- 

How much actual control do 

you have in your classroom at 

this school over the following 

areas of your planning a teaching? 

a. Selecting textbooks and other N , T , . 

. „ . , A . , (1) No control 

instructional materials ... . 

Classroom „ . b. Selecting content, topics, and ^ A , in ,° r C + ° n r ° 

Control 054 skills to be taught. 3 Moderate 

c. Selecting teaching techniques. ^ rea Ca ° 

d. Evaluating and grading COn r ° 

students. 

e. Disciplining students. 

f. Detennining the amount of 
homework to be assigned. 

No 

0.722 (U); 
0.745 (S) 

To what extent do you agree or 
disagree with each of the 
following statement? 

g. My principal enforces school (1) Strongly agree 
rules for student conduct and (2) Somewhat 

T , , . backs me up when I need it. agree 

s “ ^ Q55. j. The principal knows what (3) Somewhat 

^ kind of school he or she wants disagree 

and has communicated it to the (4) Strongly 
staff. disagree 

1. In this school, staff members 
are recognized for a job well 
done. 

Yes 

0.728 (U); 
0.839 (S) 

To what extent is each of the (1) Serious 

following a problem in this problem 

Student _ , school? (2) Moderate 

Behavior v ' a. Student tardiness problem 

b. Student absenteeism (3) Minor problem 

c. Student class cutting (4) Not a problem 

Yes 

0.727 (U); 
0.728 (S) 


Notes: U and S refer to unstandardized and standardized alphas. We standardized the response scales to have within-sample mean 
0 and variance 1 after taking averages over the respective sets of scales. 
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Table A.2. State Testing Windows 


State 

Begin 

End 

Name 

State Test 
Sample 

AL 

March 31, 2008 

April 11,2008 

Alabama Reading and Mathematics Test 

No 

AK 

March 31, 2008 

April 14, 2008 

Standards Based Assessment 

No 

AZ 

April 7, 2008 

April 18,2008 

Arizona Instrument to Measure Standards (AIMS) 

No 

AR 

April 14, 2008 

April 18,2008 

Benchmark Exams 

No 

CA 

April 15, 2008 

April 28, 2008 

California Achievement Test CAT/6 

No 

CO 

March 10, 2008 

April 11,2008 

Colorado Student Assessment Program 

No 

CT 

March 3, 2008 

March 31, 2008 

Connecticut Mastery/Connecticut Academic Perfonnance Test 

No 

DE 

March 5, 2008 

March 14, 2008 

Delaware Student Testing Program 

No 

DC 

April 22, 2008 

May 2, 2008 

Stanford Achievement Test 

No 

FL 

March 12, 2008 

March 25, 2008 

Florida Comprehensive Assessment Test 

No 

GA 

April 2, 2008 

May 2, 2008 

Criterion Referenced Competency Tests 

No 

HI 

March 31, 2008 

April 18,2008 

Hawaii Content and Perfonnance Standards II (HCPS) 

No 

ID 

April 14, 2008 

May 16, 2008 

Idaho Standards Achievement Tests 

No 

IL 

March 3, 2008 

March 14, 2008 

Illinois Standards Achievement Test 

No 

IN 

March 3, 2008 

March 13, 2008 

Indiana Statewide Testing for Educational Progress-Plus 





(ISTEP) 

No 

IA 

October 22, 2007 

October 26, 2007 

Iowa Tests of Basic Skills (ITBS) 

Yes 

KS 

February 18, 2008 

April 14, 2008 

Kansas State Assessment 

No 

KY 

April 21, 2008 

May 2, 2008 

Kentucky Core Content Test 

No 

LA 

March 10, 2008 

March 14, 2008 

Louisiana Educational Assessment Program (LEAP) 

No 

ME 

March 3, 2008 

March 21, 2008 

Maine Educational Assessment 

No 

MD 

April 1,2008 

April 10, 2008 

Maryland School Assessment 

No 

MA 

March 24, 2008 

April 4, 2008 

Massachusetts Comprehensive Assessment System 

No 

MI 

October 8, 2007 

October 26, 2007 

Michigan Educational Assessment Program 

Yes 

MN 

April 14, 2008 

May 2, 2008 

Minnesota Comprehensive Assessments - Series II 

No 

MS 

May 13, 2008 

May 14, 2008 

Mississippi Curriculum Tests 2 

No 

MO 

March 31, 2008 

April 25, 2008 

Missouri Assessment Program 

No 

MT 

March 3, 2008 

March 26, 2008 

Criterion-Referenced Tests (CRT) 

No 

NE 

January 28, 2008 

February 8, 2008 

STARS Writing Assessments in grades 

No 

NV 

January 22, 2008 

February 22, 2008 

Iowa Tests of Basic Skills/Iowa Tests of Educational 

No 


25 







Timing Is Everything 


State 

Begin 

End 

Name 

State Test 
Sample 




Development 


NH 

May 5, 2008 

May 22, 2008 

New England Common Assessment 

No 

NJ 

March 10, 2008 

May 8, 2008 

New Jersey Skills and Knowledge Assessment (NJ ASK) 

No 

NM 

February 25, 2008 

March 21,2008 

The New Mexico Standards Based Assessment 

No 

NY 

March 3, 2008 

March 12, 2008 

Mathematics Assessment Tests 

No 

ND 

October 22, 2007 

November 9, 

North Dakota State Assessment 




2007 


Yes 

OH 

April 21, 2008 

May 9, 2008 

Ohio Achievement Test 

No 

OK 

April 10, 2008 

April 25, 2008 

Oklahoma Core Curriculum Tests 

No 

OR 

October 15, 2007 

May 23, 2008 

Knowledge and Skills Tests 

No 

PA 

March 31, 2008 

April 11,2008 

Pennsylvania System of School Assessment 

No 

RI 

October 1, 2007 

October 23, 2007 

New England Common Assessment 

Yes 

SC 

May 13, 2008 

May 22, 2008 

Palmetto Achievement Challenge Test 

No 

SD 

April 2, 2008 

April 20, 2008 

Dakota State Testing of Educational Progress 

No 

TN 

March 31, 2008 

April 23, 2008 

Tennessee Comprehensive Assessment Program Achievement 





Test 

No 

TX 

March 3, 2008 

March 8, 2008 

Texas Assessment of Knowledge and Skills 

No 

UT 

September 18, 2008 

October 5, 2008 

Iowa Tests (/Utah Core Curriculum) 

Yes 

VT 

October 2, 2007 

October 19, 2007 

The New England Common Assessment Program 

Yes 

VA 

April 14, 2008 

June 13, 2008 

SOL Multiple Choice 

No 

WA 

April 14, 2008 

May 2, 2008 

Washington Assessment of Student Learning 

No 

WV 

May 12, 2008 

May 16, 2008 

West Virginia Educational Standards Tests 

No 

WI 

October 22, 2007 

November 23, 

Wisconsin Knowledge and Concepts Examinations 




2007 


Yes 

WY 

March 12, 2008 

April 16, 2008 

Proficiency Assessments for Wyoming Schools (PAWS) 

No 


Notes: States offering a large-scale test during fall semester were included in our State Test Sample. Oregon was excluded because of a wide range of testing window period. 
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Table A.3. Descriptive Statistics: Scale Means for Poverty Related Challenges by Day, 
Season, and Testing Period 


Temporal 

variation 


Average teachers’ perceptions: Student poverty 


Full Sample: 








Day of the week 

Sun 

Mon 

Tue 

Wed 

Thu 

Fri 

Sat 


-0.121 

-0.059 

-0.025 

-0.038 

-0.038 

-0.006 

-0.008 


(0.054) 

(0.031) 

(0.029) 

(0.033) 

(0.034) 

(0.031) 

(0.065) 

Season 

Fall 

Winter 

Spring 






-0.100 

0.061 

0.015 






(0.024) 

(0.022) 

(0.084) 





Pre- & post-testing 

Prior 

Post 







-0.051 

-0.067 







(0.021) 

(0.044) 






State Test Sample: 








Day of the week 

Sun 

Mon 

Tue 

Wed 

Thu 

Fri 

Sat 


-0.141 

-0.197 

-0.142 

-0.151 

-0.053 

- 0.111 

-0.134 


(0.138) 

(0.113) 

(0.096) 

(0.087) 

(0.105) 

(0.098) 

(0.156) 

Season 

Fall 

Winter 

Spring 






-0.135 

-0.128 

-0.237 






(0.061) 

(0.103) 

(0.233) 





Pre- & post-testing 

Prior 

Post 







-0.447 

-0.091 







(0.135) 

(0.057) 







Note: All data weighted using adjusted the balanced repeated replication (BRR) weights and the standard deviation are presented 
in parentheses. 
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Table A.4. Descriptive Statistics: Scale Means for Classroom Control by Day, Season, and 
Testing Period 


Temporal 

variation 


Average teachers’ perceptions: Classroom control 


Full Sample: 








Day of the week 

Sun 

Mon 

Tue 

Wed 

Thu 

Fri 

Sat 


-0.282 

-0.230 

-0.180 

-0.226 

-0.192 

-0.204 

-0.253 


(0.058) 

(0.029) 

(0.023) 

(0.035) 

(0.031) 

(0.025) 

(0.070) 

Season 

Fall 

Winter 

Spring 






-0.196 

-0.240 

-0.249 






(0.018) 

(0.028) 

(0.070) 





Pre- & post-testing 

Prior 

Post 







-0.231 

-0.044 







(0.018) 

(0.034) 






State Test Sample: 








Day of the week 

Sun 

Mon 

Tue 

Wed 

Thu 

Fri 

Sat 


-0.189 

-0.203 

-0.115 

-0.041 

0.011 

-0.172 

-0.029 


(0.115) 

(0.073) 

(0.067) 

(0.076) 

(0.095) 

(0.083) 

(0.149) 

Season 

Fall 

Winter 

Spring 






-0.104 

-0.118 

-0.214 






(0.043) 

(0.077) 

(0.201) 





Pre- & post-testing 

Prior 

Post 







-0.220 

-0.124 







(0.139) 

(0.041) 







Note: All data weighted using adjusted the balanced repeated replication (BRR) weights and the standard deviation are presented 
in parentheses. 
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Table A.5. Descriptive Statistics: Scale Means for Leadership Support by Day, Season, and 
Testing Period 


Temporal 

variation 


Average teachers’ perceptions: Leadership support 


Full Sample: 








Day of the week 

Sun 

Mon 

Tue 

Wed 

Thu 

Fri 

Sat 


0.140 

0.045 

0.087 

0.036 

0.060 

0.014 

0.057 


(0.044) 

(0.025) 

(0.023) 

(0.025) 

(0.027) 

(0.032) 

(0.054) 

Season 

Fall 

Winter 

Spring 






0.093 

0.013 

-0.222 






(0.015) 

(0.019) 

(0.067) 





Pre- & post-testing 

Prior 

Post 







0.078 

-0.082 







(0.013) 

(0.040) 






State Test Sample: 








Day of the week 

Sun 

Mon 

Tue 

Wed 

Thu 

Fri 

Sat 


0.016 

0.075 

0.064 

0.157 

0.051 

0.011 

-0.132 


(0.156) 

(0.083) 

(0.070) 

(0.066) 

(0.105) 

(0.099) 

(0.173) 

Season 

Fall 

Winter 

Spring 






0.107 

-0.024 

-0.371 






(0.045) 

(0.083) 

(0.332) 





Pre- & post-testing 

Prior 

Post 







0.369 

0.031 







(0.074) 

(0.055) 







Note: All data weighted using adjusted the balanced repeated replication (BRR) weights and the standard deviation are presented 
in parentheses. 
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Table A. 6. Descriptive Statistics: Scale Means for Student Behavior Problem by Day, 
Season, and Testing Period 


Temporal 

variation 


Average teachers’ perceptions: Student behavior 


Full Sample: 








Day of the week 

Sun 

Mon 

Tue 

Wed 

Thu 

Fri 

Sat 


-0.334 

-0.238 

-0.185 

-0.167 

-0.197 

-0.153 

-0.236 


(0.040) 

(0.023) 

(0.022) 

(0.023) 

(0.026) 

(0.026) 

(0.069) 

Season 

Fall 

Winter 

Spring 






-0.290 

-0.059 

-0.048 






(0.016) 

(0.024) 

(0.086) 





Pre- & post-testing 

Prior 

Post 







-0.209 

-0.183 







(0.015) 

(0.041) 






State Test Sample: 








Day of the week 

Sun 

Mon 

Tue 

Wed 

Thu 

Fri 

Sat 


-0.595 

-0.594 

-0.583 

-0.575 

-0.462 

-0.427 

-0.635 


(0.108) 

(0.074) 

(0.066) 

(0.068) 

(0.085) 

(0.066) 

(0.144) 

Season 

Fall 

Winter 

Spring 






-0.565 

-0.468 

-0.848 






(0.044) 

(0.067) 

(0.241) 





Pre- & post-testing 

Prior 

Post 







-0.400 

-0.545 







(0.060) 

(0.042) 







Note: All data weighted using adjusted the balanced repeated replication (BRR) weights and the standard deviation are presented 
in parentheses. 
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Table A. 7. Descriptive Statistics of Survey Measures: Full Sample and State Test Sample 

__ Full Sample __ State Test Sample 



Mean 



Percentiles 



Mean 








10% 

25% 

50% 

75% 

90% 


10% 

25% 

50% 

75% 

90% 

Entire Sample: 













- Student poverty 

-0.038 

(0.018) 

-1.708 

-0.665 

0.379 

0.379 

1.422 

-0.136 

(0.051) 

-1.708 

-0.665 

-0.665 

0.379 

1.422 

- Classroom control 

-0.213 

(0.017) 

-1.731 

-0.748 

-0.092 

0.564 

1.219 

-0.110 

(0.038) 

-1.404 

-0.748 

-0.092 

0.564 

1.219 

- Leadership support 

0.056 

(0.012) 

-1.506 

-0.472 

0.046 

1.081 

1.081 

0.060 

(0.047) 

-1.506 

-0.472 

0.563 

1.081 

1.081 

- Student behavior 

-0.200 

(0.013) 

-1.623 

-0.763 

-0.333 

0.528 

0.958 

-0.547 

(0.037) 

-1.623 

-1.193 

-0.763 

0.097 

0.528 

First Respondents: 













- Student poverty 

-0.020 

(0.024) 

-1.708 

-0.665 

0.379 

0.379 

1.422 

-0.130 

(.0560) 

-1.708 

-0.665 

-0.665 

0.379 

1.422 

- Classroom control 

-0.266 

(0.025) 

-1.731 

-1.075 

-0.092 

0.564 

1.219 

-0.137 

(.0493) 

-1.404 

0.748 

-0.092 

0.564 

1.219 

- Leadership support 

0.064 

(0.018) 

-1.506 

-0.472 

0.046 

1.081 

1.081 

0.098 

(.0620) 

-1.506 

-0.472 

0.563 

1.081 

1.081 

- Student behavior 

-0.294 

(0.021) 

-1.623 

-0.763 

-0.333 

0.097 

0.958 

-0.568 

(.0472) 

-1.623 

-1.193 

-0.763 

-0.333 

0.528 

Note: All data weighted using adjusted the balanced repeated replication (BRR) weights and the standard 

deviation are presented in 

parentheses. 
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Table A. 8. Descriptive Statistics—Sample Means Calculated Using Teacher Weights 



Full Sample 

State-Test Sample 


Entire Sample 

First Respondents 

Entire Sample 

First Respondents 

No. observations 

33,258 

10,548 

2,102 

831 

School Factors 





Average total enrollment 

845.09 

713.53 

473.77 

460.10 


(13.79) 

(12.22) 

(15.53) 

(17.67) 

Average percentage of teacher’s students with an IEP (%) 

15.44 

15.86 

15.17 

16.25 


(0.41) 

(0.66) 

(0.78) 

(1.17) 

Average percentage of teacher’s students with an LEP (%) 

8.42 

8.80 

4.73 

5.29 


(0.30) 

(0.55) 

(0.54) 

(0.93) 

Urbanicity (%): 





-City 

26.16 

26.05 

21.52 

21.34 

- Suburb 

34.68 

34.02 

31.77 

32.40 

- Town 

13.99 

13.99 

15.35 

15.71 

- Rural 

25.16 

25.93 

31.35 

30.55 

Region (%): 





- Northeast 

19.46 

20.03 

8.31 

8.70 

- Midwest 

22.40 

21.75 

83.21 

82.49 

- South 

39.53 

38.78 

0.00 

0.00 

- West 

18.62 

19.44 

8.47 

8.81 

Charter School (%): 

2.08 

2.07 

5.26 

4.56 

School level (%): 





- Primary school 

46.65 

55.28 

64.78 

69.47 

- Middle school 

18.84 

18.95 

24.62 

22.02 

- High school 

29.80 

21.46 

0.00 

0.00 

- Combined school 

4.71 

4.30 

10.60 

8.51 

Program type (%): 





- Regular 

93.52 

93.44 

96.93 

97.06 

- Special program emphasis 

2.90 

2.82 

1.54 

1.39 

- Special education 

0.80 

0.88 

0.73 

0.94 

- G are er/technical/vocat icnal education 

1.07 

0.81 

0.00 

0.00 

- Alternative 

1.72 

2.05 

0.80 

0.61 
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Full Sample 

State-Test Sample 

Entire Sample 

First Respondents 

Entire Sample 

First Respondents 

Teacher Factors 





Female (%) 

75.32 

77.74 

8218 

85.08 

Union (%) 

76.58 

76.47 

8864 

88.72 

Average years of teaching experience 

13.52 

13.94 

14.37 

14.42 


(0.16) 

(0.21) 

(0.47) 

(0.74) 

Ethnicity (%): 





- Hispanic ethnic origin 

7.31 

6.51 

2.30 

2.04 

Race (%): 





- White 

90.31 

91.30 

97.34 

97.95 

- Black/African American 

7.68 

6.51 

2.23 

2.02 

- Asian 

1.56 

1.61 

0.47 

0.33 

- Native Hawaiian/Pacific Islander 

0.33 

0.30 

0.00 

0.00 

- American Indian/Alaska Native 

1.20 

1.23 

0.67 

0.97 

Highest degree earned (%) 





- Associate’s 

0.76 

0.54 

0.35 

0.38 

- Bachelor’s 

48.08 

47.21 

47.52 

43.54 

- Master’s 

44.24 

45.64 

47.15 

51.66 

- Education specialist 

6.08 

5.90 

4.71 

4.07 

- Doctoral or professional 

0.84 

0.70 

0.27 

0.34 

Currently teaching grades of students (%) 





- Pre-kindergarten 

2.01 

2.20 

4.38 

3.98 

- Kindergarten 

12.81 

14.47 

19.86 

16.97 

- 1st 

15.02 

16.35 

21.55 

21.38 

- 2nd 

14.96 

16.36 

21.00 

19.99 

- 3rd 

15.32 

17.31 

21.49 

20.40 

- 4th 

14.85 

16.53 

20.24 

18.43 

- 5th 

14.71 

16.90 

21.47 

21.03 

- 6th 

15.84 

16.92 

24.59 

21.87 

- 7th 

16.81 

16.47 

21.87 

19.79 

- 8th 

17.11 

17.02 

22.18 

17.28 

- 9th 

23.67 

17.73 

6.35 

4.20 

- 10th 

26.75 

19.71 

6.46 

4.38 

- 11th 

27.14 

19.86 

6.78 

4.65 

- 12 th 

26.15 

19.23 

6.71 

4.65 

- Ungraded 

3.38 

2.82 

1.56 

1.79 


Note: All data weighted using adjusted the balanced repeated replication (BRR) weights and the standard deviation are presented in parentheses. 
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Appendix B. Regression Outcomes 


Table B.l. Variation across Days of Week 



Poverty Challenges 
(1) (2) 

Classroom Control 
(1) (2) 

Leadership Support 
(1) (2) 

Student Behavior 
(1) (2) 

Sunday 

-0.115+ 

-0.061 

-0.078 

-0.007 

0.126* 

0.117+ 

-0.181*** 

-0.065 


(0.063) 

(0.054) 

(0.064) 

(0.056) 

(0.053) 

(0.060) 

(0.047) 

(0.042) 

Monday 

-0.053 

-0.084* 

-0.026 

-0.040 

0.031 

0.042 

-0.085* 

-0.067* 


(0.040) 

(0.037) 

(0.037) 

(0.032) 

(0.039) 

(0.041) 

(0.032) 

(0.031) 

Tuesday 

-0.019 

-0.029 

0.024 

0.011 

0.073+ 

0.088* 

-0.032 

-0.042 


(0.040) 

(0.041) 

(0.032) 

(0.036) 

(0.039) 

(0.041) 

(0.032) 

(0.028) 

Wednesday 

-0.032 

-0.054 

-0.022 

-0.007 

0.021 

0.041 

-0.014 

-0.040 


(0.043) 

(0.043) 

(0.043) 

(0.036) 

(0.038) 

(0.039) 

(0.034) 

(0.035) 

Thursday 

-0.032 

-0.030 

0.012 

0.007 

0.045 

0.075 

-0.044 

-0.052 


(0.043) 

(0.045) 

(0.041) 

(0.042) 

(0.045) 

(0.046) 

(0.036) 

(0.032) 

Saturday 

-0.002 

0.034 

-0.049 

0.005 

0.042 

0.037 

-0.084 

0.015 


(0.068) 

(0.063) 

(0.071) 

(0.063) 

(0.060) 

(0.067) 

(0.068) 

(0.067) 

School controls 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

Teacher controls 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

R-squared 

No. of 

0.001 

33,258 

0.118 

29,260 

0.001 

33,258 

0.164 

29,260 

0.001 

33,258 

0.030 

29,260 

0.002 

33,258 

0.256 

29,260 


observations _ 

Notes: The outcomes of OLS regression models are reported. The endogenous explanatory variables for school and teacher characteristics are included where indicated but not 
reported above. The standard errors are in parentheses. The model is fitted using the BRR weights. The symbols +, *, **, and *** indicate statistical significance at the 10%, 5%, 
1%, 0.1% levels, respectively. 
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Table B.2. Variation across Days of Week: First and Later Respondents 



Poverty Challenges 
First Later 

(1) (2) 

Classroom Control 

First Later 

(1) (2) 

Leadership Support 

First Later 

(1) (2) 

Student Behavior 

First Later 

(1) (2) 

Sunday 

-0.061 

-0.059 

-0.046 

0.006 

0.266* 

0.062 

-0.085 

-0.058 


(0.116) 

(0.061) 

(0.111) 

(0.061) 

(0.121) 

(0.082) 

(0.091) 

(0.047) 

Monday 

-0.105 

-0.075 

-0.052 

-0.031 

0.060 

0.039 

-0.103 

-0.051 


(0.082) 

(0.051) 

(0.063) 

(0.038) 

(0.067) 

(0.054) 

(0.064) 

(0.032) 

Tuesday 

-0.016 

-0.044 

0.050 

-0.014 

0.133+ 

0.068 

-0.077 

-0.023 


(0.081) 

(0.053) 

(0.066) 

(0.046) 

(0.073) 

(0.053) 

(0.058) 

(0.036) 

Wednesday 

-0.040 

-0.062 

-0.019 

0.005 

0.063 

0.033 

-0.062 

-0.028 


(0.082) 

(0.053) 

(0.067) 

(0.047) 

(0.080) 

(0.057) 

(0.064) 

(0.038) 

Thursday 

-0.017 

-0.040 

-0.025 

0.024 

0.177* 

0.023 

-0.076 

-0.041 


(0.075) 

(0.055) 

(0.083) 

(0.049) 

(0.083) 

(0.064) 

(0.069) 

(0.033) 

Saturday 

0.103 

0.001 

0.072 

-0.029 

0.123 

-0.009 

-0.103 

0.070 


(0.127) 

(0.079) 

(0.129) 

(0.076) 

(0.119) 

(0.089) 

(0.113) 

(0.073) 

School controls 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Teacher controls 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

R-squared 

No. of 

0.121 

9,153 

0.123 

20,107 

0.177 

9,153 

0.162 

20,107 

0.039 

9,153 

0.030 

20,107 

0.233 

9,153 

0.265 

20,107 


observations _ 

Notes: The outcomes of OLS regression models are reported. The endogenous explanatory variables for school and teacher characteristics are included where indicated but not 
reported above. The standard errors are in parentheses. The model is fitted using the BRR weights. The symbols +, *, **, and *** indicate statistical significance at the 10%, 5%, 
1%, 0.1% levels, respectively. 
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Table B.3. Poverty Related Challenges over the School Year 



OLS 

OLS 

OLS 

OLS 

IV 

IV 

Sample 

All 

All 

First 

Later 

All 

All 


0) 

(2) 

(3) 

(4) 

(5) 

(6) 

Days 

0.001 

0.001 

0.002 

0.001 

0.002 

0.002 


(0.001) 

(0.001) 

(0.001) 

(0.001) 

(0.002) 

(0.002) 

Winter 

0.062 

0.078 

0.067 

0.072 

0.048 

-0.019 


(0.077) 

(0.069) 

(0.099) 

(0.082) 

(0.123) 

(0.118) 

Spring 

-0.102 

-0.010 

-0.408 

0.068 

-0.132 

-0.225 


(0.166) 

(0.161) 

(0.269) 

(0.183) 

(0.273) 

(0.266) 

School controls 

No 

Yes 

Yes 

Yes 

No 

Yes 

Teacher controls 

No 

Yes 

Yes 

Yes 

No 

Yes 

R-squared 

0.006 

0.120 

0.125 

0.126 



No. of observations 

33,258 

29,260 

9,153 

20,107 

33,258 

29,260 


Notes: The outcomes of OLS (columns 1-4) and 2SLS (columns 5-6) regression models are reported. The endogenous explanatory variables for school and teacher characteristics 
are included but not reported above. The standard errors are in parentheses. The model is fitted using the BRR weights. The symbols +, *, **, and *** indicate statistical 
significance at the 10%, 5%, 1%, 0.1% levels, respectively. 
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Table B.4. Classroom Control over the School Year 


OLS 

OLS 

OLS 

OLS 

IV 

IV 

Sample 

All 

All 

First 

Later 

All 

All 


0) 

(2) 

(3) 

(4) 

(5) 

(6) 

Days 

-0.001 

0.000 

0.001 

0.000 

-0.004* 

0.001 


(0.001) 

(0.001) 

(0.001) 

(0.001) 

(0.002) 

(0.002) 

Winter 

0.032 

-0.031 

-0.008 

-0.036 

0.218+ 

-0.089 


(0.069) 

(0.055) 

(0.085) 

(0.069) 

(0.118) 

(0.106) 

Spring 

0.113 

-0.057 

-0.016 

-0.062 

0.522* 

-0.185 


(0.168) 

(0.146) 

(0.212) 

(0.177) 

(0.237) 

(0.229) 

School controls 

No 

Yes 

Yes 

Yes 

No 

Yes 

Teacher controls 

No 

Yes 

Yes 

Yes 

No 

Yes 

R-squared 

0.001 

0.164 

0.176 

0.162 



No. of observations 

33,258 

29,260 

9,153 

20,107 

33,258 

29,260 

Notes: The outcomes of OLS (columns 1-4) and 2SLS (columns 5-6) regression models 

are reported. The endogenous explanatory variables for school and teacher characteristics 

are included but not reported above. The standard errors are in 

parentheses. The model i 

is fitted using the BRR weights. The symbols +, *. 

^ •I* £t,DCl 

indicate statistical 

significance at the 10%, 5%, 1% 

, 0.1% levels, respectively. 






Table B.5. Leadership Support over the School Year 






OLS 

OLS 

OLS 

OLS 

IV 

IV 

Sample 

All 

All 

First 

Later 

All 

All 


(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

Days 

-0.002* 

-0.001+ 

-0.001 

-0.002+ 

-0.000 

0.000 


(0.001) 

(0.001) 

(0.001) 

(0.001) 

(0.001) 

(0.001) 

Winter 

0.027 

0.040 

0.066 

0.038 

-0.062 

-0.061 


(0.050) 

(0.055) 

(0.091) 

(0.063) 

(0.087) 

(0.090) 

Spring 

-0.080 

-0.034 

0.245 

-0.092 

-0.274 

-0.258 


(0.119) 

(0.138) 

(0.222) 

(0.158) 

(0.191) 

(0.209) 

School controls 

No 

Yes 

Yes 

Yes 

No 

Yes 

Teacher controls 

No 

Yes 

Yes 

Yes 

No 

Yes 

R-squared 

0.004 

0.031 

0.035 

0.035 



No. of observations 

33,258 

29,260 

9,153 

20,107 

33,258 

29,260 


Notes: The outcomes of OLS (columns 1-4) and 2SLS (columns 5-6) regression models are reported. The endogenous explanatory variables for school and teacher characteristics 
are included but not reported above. The standard errors are in parentheses. The model is fitted using the BRR weights. The symbols +, *, **, and *** indicate statistical 
significance at the 10%, 5%, 1%, 0.1% levels, respectively. 
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Table B.6. Student Behavior Problems over the School Year 



OLS 

OLS 

OLS 

OLS 


IV 

IV 

Sample 

All 

All 

First 

Later 


All 

All 


(1) 

(2) 

(3) 

(4) 


(5) 

(6) 

Days 

0.004*** 

0.003*** 

0.003** 

0.003** 

0.003* 

0.004** 


(0.001) 

(0.001) 

(0.001) 

(0.001) 

(0.001) 

(0.001) 

Winter 

-0.041 

-0.031 

-0.034 

-0.039 

0.038 

-0.116 


(0.059) 

(0.055) 

(0.082) 

(0.062) 

(0.091) 

(0.087) 

Spring 

-0.355* 

-0.185 

-0.547** 

-0.108 

- 

0.181 

-0.375+ 


(0.147) 

(0.139) 

(0.194) 

(0.156) 

(0.205) 

(0.194) 

School controls 

No 

Yes 

Yes 

Yes 


No 

Yes 

Teacher controls 

No 

Yes 

Yes 

Yes 


No 

Yes 

R-squared 

0.019 

0.263 

0.242 

0.272 




No. of observations 

33,258 

29,260 

9,153 

20,107 

33,258 

29,260 

Notes: The outcomes of OLS (columns 1-4) and 2SLS (columns 5-6) regression models are reported. The endogenous explanatory variables for school and teacher characteristics 

are included but not reported above. The standard errors are in parentheses. The model is fitted using the BRR weights. The symbols +, *, * N 

! , and *** indicate statistical 

significance at the 10%, 5%, 1%, 0.1% levels, respectively. 







Table B. 7. Variation before and after State Exams 







Poverty Challenges 

Classroom Control 

Leadership Support 

Student Behavior 


0) 

(2) 

0) (2) 

(1) 

(2) 

(1) 

(2) 

Time prior 

-0.001 

-0.001 

-0.001+ -0.000 

-0.001 

-0.001 

0.001 

-0.001 


(0.001) 

(0.001) 

(0.001) (0.001) 

(0.001) 

(0.001) 

(0.001) 

(0.001) 

Prior 

-0.105 

-0.038 

0.181 0.347+ 

0.545** 

0.238 

-0.045 

-0.147 


(0.219) 

(0.342) 

(0.214) (0.187) 

(0.196) 

(0.282) 

(0.165) 

(0.158) 

School controls 

No 

Yes 

No Yes 

No 

Yes 

No 

Yes 

Teacher controls 

No 

Yes 

No Yes 

No 

Yes 

No 

Yes 

R-squared 

0.017 

0.157 

0.006 0.243 

0.014 

0.090 

0.007 

0.157 

No. of observations 

1,829 

1,487 

1,829 1,487 

1,829 

1,487 

1,829 

1,487 

Notes: The outcomes of OLS regression models are reported. The endogenous explanatory variables for school and teacher characteristics are included but not reported above. The 

standard errors in parentheses 

are in parentheses. The model 

is fitted using the BRR weights. The symbols +, *, 

sjc s]c 

indicate statistical significance at the 10%, 5%, 1%, 


0.1% levels, respectively. 
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Table B.8. Variation before and after State Exams: Response Time 



Poverty Challenges 

Classroom Control 

Leadership Support 

Student Behavior 


First 

Later 

First 

Later 

First 

Later 

First 

Later 


(1) 

(2) 

(1) 

(2) 

(1) 

(2) 

(1) 

(2) 

Time prior 

-0.002 

-0.001 

- 0.000 

0.001 

0.000 

-0.002 

-0.001 

-0.001 


(0.003) 

(0.001) 

(0.001) 

(0.001) 

(0.002) 

(0.001) 

(0.002) 

(0.001) 

Prior 

0.053 

-0.530+ 

0.304 

0.082 

0.029 

0.847* 

-0.086 

-0.408+ 


(0.486) 

(0.310) 

(0.299) 

(0.481) 

(0.289) 

(0.365) 

(0.209) 

(0.237) 

School controls 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Teacher controls 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

R-squared 

0.194 

0.185 

0.249 

0.298 

0.187 

0.115 

0.219 

0.175 

No. of observations 

573 

914 

573 

914 

573 

914 

573 

914 


Notes: The outcomes of OLS regression models are reported. The endogenous explanatory variables for school and teacher characteristics are included but not reported above. The 
standard errors in parentheses are in parentheses. The model is fitted using the BRR weights. The symbols +, *, **, and *** indicate statistical significance at the 10%, 5%, 1%, 
0.1% levels, respectively. 
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